Systems and Methods for Force-Based Object Manipulation and Haptic Sensations

ABSTRACT

Systems and methods for force-based object manipulation and haptic sensations are disclosed. One disclosed method includes the steps of receiving a first signal indicating a location of a user interaction and receiving a second signal indicating a first force. The method also includes, if the location of the user interaction corresponds to an object displayed on a display screen: outputting a first haptic signal to a haptic output device to cause a first haptic effect; and outputting a second haptic signal to the haptic output device to cause a second haptic effect if the first force meets or exceeds a first force threshold.

REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 14/663,406, entitled “Systems and Methods forForce-Based Object Manipulation and Haptic Sensations,” filed Mar. 19,2015, which claims priority to U.S. Provisional Patent Application No.61/968,913, entitled “Systems and Methods for Pressure-Based ObjectManipulation and Haptic Sensations,” filed Mar. 21, 2014, the entiretyof each of which is hereby incorporated by reference herein.

FIELD

The present disclosure generally relates to force-based haptic methodsand devices, and more specifically relates to systems and methods forforce-based object manipulation and haptic sensations.

BACKGROUND

Many modern processor-based devices are equipped with touch-sensitiveinput devices, such as touch screens, and some may also be capable ofoutputting haptic effects. Such devices allow a user to interact withgraphically-displayed objects, such as icons or widgets to performcertain functions or to launch one or more applications. When a userinteracts with the touch screen to perform these tasks, the device mayoutput a haptic effect when the user touches such an icon or widget toindicate that the user has activated that object.

SUMMARY

Systems and methods for force-based object manipulation and hapticsensations are disclosed. For example, one disclosed method comprisesreceiving a first signal indicating a location of a user interaction;and receiving a second signal indicating a first force. The method alsocomprises, if the location of the user interaction corresponds to anobject displayed on a display screen: outputting a first haptic signalto a haptic output device to cause a first haptic effect; and outputtinga second haptic signal to the haptic output device to cause a secondhaptic effect if the first force meets or exceeds a first forcethreshold. In some examples a computer-readable medium may be encodedwith program code to cause a processor to perform such a method.

These illustrative examples are mentioned not to limit or define thescope of this disclosure, but rather to provide examples to aidunderstanding thereof Illustrative examples are discussed in theDetailed Description, which provides further description. Advantagesoffered by various examples may be further understood by examining thisspecification.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated into and constitute apart of this specification, illustrate one or more examples and,together with the description of examples, serve to explain theprinciples and implementations of systems and methods for force-basedobject manipulation and haptic sensations.

FIGS. 1A-B show systems for force-based object manipulation and hapticsensations according to examples of the present disclosure;

FIGS. 2A-B show methods for force-based object manipulation and hapticsensations according to examples of the present disclosure;

FIG. 2C shows state diagrams for force-based object manipulation andhaptic sensations according to examples of the present disclosure;

FIG. 3 shows example graphical user interfaces usable with examples ofsystems and methods for force-based object manipulation and hapticsensations according to the present disclosure;

FIG. 4 shows an example graph of forces associated with user interactionwith a force-sensitive surface according to examples of the presentdisclosure;

FIG. 5A shows a method for force-based object manipulation and hapticsensations according to examples of the present disclosure;

FIG. 5B shows a state diagram for force-based object manipulation andhaptic sensations according to examples of the present disclosure;

FIG. 6 shows an example graph of forces associated with user interactionwith a force-sensitive surface according to examples of the presentdisclosure;

FIG. 7 shows a method for force-based object manipulation and hapticsensations according to examples of the present disclosure;

FIGS. 8A-B show software applications for providing force-based objectmanipulation and haptic sensations are shown according to examples ofthis disclosure; and

FIG. 9 shows a method for force-based object manipulation and hapticsensations according to examples of the present disclosure.

DETAILED DESCRIPTION

Examples are described herein in the context of systems and methods forforce-based object manipulation and haptic sensations. Those of ordinaryskill in the art will realize that the following description isillustrative only and is not intended to be in any way limiting. Otherexamples will readily suggest themselves to such skilled persons havingthe benefit of this disclosure. Reference will now be made in detail toimplementations of examples as illustrated in the accompanying drawings.The same reference indicators will be used throughout the drawings andthe following description to refer to the same or like items.

In the interest of clarity, not all of the routine features of theimplementations described herein are shown and described. It will, ofcourse, be appreciated that in the development of any such actualimplementation, numerous implementation-specific decisions must be madein order to achieve the developer's specific goals, such as compliancewith application- and business-related constraints, and that thesespecific goals will vary from one implementation to another and from onedeveloper to another.

Illustrative System for Force-Based Object Manipulation and HapticSensations

FIG. 1A shows an illustrative system for force-based object manipulationand haptic sensations. In the example shown in FIG. 1A, the systemincludes a tablet computer 100 that has a touch-sensitive display 120(also referred to as a “touch screen”) in a housing 110 and a hapticoutput device (not shown) that is capable of simulating textures when auser contacts the touch-sensitive display 120. In addition, the tablet100 includes another haptic output device (not shown) that is capable ofoutputting vibrational effects to the touch-sensitive display 120. Thetablet 100 provides a graphical user interface (GUI) to allow the userto interact with the tablet 100, such as by launching applications,manipulating objects or images displayed on the screen, or otherwiseperforming various tasks. In this example, since the tablet 100 isenabled with the touch-sensitive display 120, the user may elect totouch the touch-sensitive display 120 to select icons, perform gesturesto provide inputs to the GUI provided by the tablet's operating system(OS) or by other GUIs available through applications, widgets, or othertypes of software.

In addition, the tablet 100 is equipped with one or more force sensorsthat sense the force applied to the touch-sensitive display 120, such aswhen the user uses her finger to contact the touch-sensitive display120. In this example, the touch-sensitive display 120 has multi-touchcapability. That is, the touch-sensitive display 120 is capable ofinterpreting multiple simultaneous (or substantially simultaneous)contacts and corresponding forces to allow the user to perform gesturesusing multiple fingers (or other input devices, such as styluses). Asthe user touches the touch-sensitive display 120, the touch-sensitivedisplay 120 and force sensors send location and force information to aprocessor of the tablet 100, which is configured to take certain actionsbased on the location and force of the user's contact.

For example, the tablet 100 may display arrangements of icons in a GUIthat represent different applications or widgets installed on the tablet100. In some cases, a user may wish to interact with the touch-sensitivedisplay 120 without activating or otherwise interacting with componentsof the GUI. For example, the tablet 100 is configured to allow a user tolightly contact the touch-sensitive display 120 and receive hapticfeedback based on the contact without activating an icon or widget, orotherwise manipulating the GUI. Thus, the user could drag her fingeracross the touch-sensitive display 120 to feel where different icons orwidgets are located, based on various haptic effects, and differentiatebetween the different icons or widgets based on those haptic effectswithout needing to look at the touch-sensitive display 120. Suchfunctionality may be particularly advantageous in situations where userattention is focused elsewhere, such as while the user is driving, butattempting to change radio stations or adjust climate control settings.If the user elects to select an icon or widget, or to otherwisemanipulate the GUI (e.g. to swipe to another page of icons), the userwould press harder on the touch-sensitive display 120 and perform thedesired gesture. By increasing the force on the touch-sensitive display120, the tablet 100 changes from an “exploration” mode, in which theuser can explore the screen tactilely, to a “manipulation” mode, inwhich the user can activate icons or widgets, or otherwise manipulatethe GUI. In some examples, the user can return to the exploration modeby reducing the force on the touch-sensitive display 120.

To enable this functionality, the tablet 100 includes software todetermine force thresholds, which may be customized by the user or maybe automatically tuned over time by the device. Such tuning may occurtransparently or interactively, such as by presenting a message to theuser—e.g., “launching application . . . did you intend to launch theapp?”—and providing the user an option to select either “yes” or “no.”Such information may be fed back into an adaptive determination of forcethresholds. Thus, by comparing the force of the user's contacts with theforce threshold(s), the device can switch between exploration andmanipulation modes. In addition, the device may provide other types offeedback to indicate the mode or the change in modes, such as additionalhaptic effects; visual effects, such as pop-up messages or changes inthe display (e.g., by greying out features while in exploration mode);or audible effects, like chimes or chords.

In addition to the exploration and manipulation modes, the tablet 100also includes a second force threshold, above which manipulations of theGUI change in nature. When the user is pressing particularly hard, todissuade such an amount of force, the GUI may become less responsive tomanipulation. In such a case, icons may move more slowly, as if they arebeing dragged through a viscous material or are experiencing frictionalresistance based on a virtual surface below the icon, or applicationicons may initially slide out from under the user's finger rather thanbeing easily selectable. In addition, upon the user's contact forcemeeting or exceeding the second threshold, the tablet 100 may output ahaptic effect to the user to indicate that the second force thresholdhas been reached. Similarly, as the user reduces the force below thesecond force threshold, another haptic effect may be output to confirmto the user that the force level is appropriate for manipulating theGUI. In some examples discussed in more detail below, a threshold mayhave two different values, one that is used as force is being increased,and another as the force is being decreased. Or, in some examples, theforce must drop below a threshold (including in the case of a hystereticthreshold) for a minimum duration before the mode switches. In otherwords, some forces may be hysteretic or “sticky.” This may provide amore user-friendly interface in cases where the user unintentionallyreduces force during a manipulation, as the user will not be frustratedif the mode of interaction unexpectedly changes mid-manipulation.

In addition to the multi-mode force interface discussed above, thetablet 100 also responds to force-based gestures to interact withvarious user interfaces. For example, the main GUI screen of the tablet100 may comprise a number of icons representing applications arranged ina tiled format. To view additional screens of the user interface, theuser may swipe to the left or right along the touch-sensitive display120 to reveal other screens with additional applications. However, theuser interface also allows the user to increase the force the userapplies to the touch-sensitive display 120 to “zoom” into the interface.For example, a user may navigate into a folder shown on the interface bysimply pressing harder on the touch-sensitive display 120. As anotherexample, if the user is interacting with a mapping application, the usermay increase or reduce the zoom level by increasing or reducing force.In some examples, the user may opt to change the type of informationdisplayed at a particular user interface level. For instance, the usermay customize the type of information displayed at a particular zoomlevel, such as from a road and highway line map to a topographical map,a satellite image, or a street view of the visible portion of the map.Thus, the use of force may enhance the number of degrees of freedom inwhat otherwise appears to be a two-dimensional interface.

The tablet 100 also employs force sensing to provide more immersivehaptic sensations to a user based on her interactions with the device.As discussed above, the main GUI screen of the user interface mayinclude a number of icons arranged in a tiled format. As a user lightlydrags her finger over the touch-sensitive display 120, she may feel ahaptic effect, such as electrostatic friction effects, as her fingerslides across an icon to indicate she is “hovering” over the icon. Toobtain more informative haptic sensations, the user may increase thecontact force. As the contact force increases, the device providesincreasingly detailed haptic sensations, such as a texture effect thatrelates to the application represented by the icon. For example, if theuser begins pressing on an icon for a calendar application, the tablet100 outputs a haptic texture that feels like a grid, to represent theshape of a monthly calendar. Or if the user begins pressing on an iconto make a phone call, the tablet 100 outputs a texture that correspondsto the shape of a telephone handset, such as by using a haptic outputdevice that deforms the contact surface on the touch-sensitive display120. In some cases, a vibrational haptic effect may be used. Forexample, if a user attempts to locate an icon to make an emergency callwhile the tablet 100 is locked, the tablet 100 outputs a repeatedsequence of haptic effects to mimic the Morse code sequence forSOS—i.e., three short vibrations followed by three longer vibrationsfollowed by three short vibrations—to indicate to the user that she haslocated the correct icon.

Thus, the illustrative device shown in FIG. 1A provides a full-featuredforce-sensitive haptic user interface to allow the user greater controland sensory feedback during her exploration and manipulation of the userinterface. These illustrative examples are given to introduce the readerto the general subject matter discussed herein and the disclosure is notlimited to these examples. The following sections describe variousadditional non-limiting examples of systems and methods for force-basedobject manipulation and haptic sensations.

Referring now to FIG. 1B, FIG. 1B shows a system 200 for force-basedobject manipulation and haptic sensations according to one example ofthis disclosure. In the example shown in FIG. 1B, the system 200comprises a housing 210, a processor 220, a memory 230, atouch-sensitive display 250, a haptic output device 240, a communicationinterface 260, a speaker 270, and a force sensor 290. In addition, thesystem 200 is in communication with haptic output device 280, which maybe optionally coupled to or incorporated into some examples. Theprocessor 220 is in communication with the memory 230 and, in thisexample, both the processor 220 and the memory 230 are disposed withinthe housing 210.

The touch-sensitive display 250, which comprises or is in communicationwith a touch-sensitive surface, is partially disposed within the housing210 such that at least a portion of the touch-sensitive display 250 isexposed to a user of the system 200. In some examples, thetouch-sensitive display 250 may not be disposed within the housing 210.For example, the system 200 may be connected to or otherwise incommunication with a touch-sensitive display 250 disposed within aseparate housing. In some examples, the housing 210 may comprise twohousings that may be slidably coupled to each other, pivotably coupledto each other or releasably coupled to each other. In still otherexamples, the system 200 may comprise or be in communication with adisplay and may comprise or be in communication with other user inputdevices, such as a mouse, a keyboard, buttons, knobs, slider controls,switches, wheels, rollers, joysticks, other manipulanda, or acombination thereof.

In some examples, one or more touch-sensitive surfaces may be includedon or disposed within one or more sides of the system 200. For example,a touch-sensitive surface may be disposed within or comprise a rearsurface of the system 200. In another example, a first touch-sensitivesurface is disposed within or comprises a rear surface of the system 200and a second touch-sensitive surface is disposed within or comprises aside surface of the system 200. In some examples, the system 200 maycomprise two or more housing components, such as in a clamshellarrangement or in a slideable arrangement. For instance, a system 200may have a clamshell configuration with a touch-sensitive display 120disposed in each of the portions of the clamshell. Furthermore, in someexamples, the touch-sensitive display 250 may not comprise atouch-sensitive surface (e.g., the touch-sensitive display 250 may be adisplay without touch-input capability). In some examples, one or moretouch-sensitive surfaces may be flexible or deformable. In otherexamples, one or more touch-sensitive surfaces may be rigid. The system200 may comprise both flexible and rigid touch-sensitive surfaces.

In the example shown in FIG. 1B, the touch-sensitive display 250 is incommunication with the processor 220 and is configured to providesignals to the processor 220 or the memory 230 and to receive signalsfrom the processor 220 or memory 230. The memory 230 is configured tostore program code, data, or both for use by the processor 220. Theprocessor 220 is configured to execute program code stored in memory 230and to transmit signals to and receive signals from the touch-sensitivedisplay 250. In the example shown in FIG. 1B, the processor 220 is alsoin communication with the communication interface 260. The processor 220is configured to receive signals from the communication interface 260and to output signals to the communication interface 260 to communicatewith other components or devices, such as one or more remote computersor servers.

In addition, the processor 220 is in communication with haptic outputdevice 240 and haptic output device 280, and is further configured tooutput signals to cause haptic output device 240, haptic output device280, or both to output one or more haptic effects. Furthermore, theprocessor 220 is in communication with speaker 270 and is configured tooutput signals to cause speaker 270 to output sounds. In variousexamples, the system 200 may comprise or be in communication with feweror additional components or devices. For example, other user inputdevices such as a mouse, trackball, trackpad, keyboard, an additionaltouch-sensitive device, or any combination of these may be comprisedwithin the system 200 or be in communication with the system 200. Asanother example, system 200 may comprise and/or be in communication withone or more accelerometers, gyroscopes, digital compasses, and/or othersensors.

The housing 210 of the system 200 shown in FIG. 1B provides protectionfor at least some of the components system 200. For example, the housing210 may be a plastic casing that protects the processor 220 and memory230 from foreign articles such as rain. In some examples, the housing210 protects the components in the housing 210 from damage if the system200 is dropped by a user. The housing 210 can be made of any suitablematerial including but not limited to plastics, rubbers, or metals.Various examples may comprise different types of housings or a pluralityof housings. For example, in some examples, the system 200 may be aportable device, handheld device, toy, trackball, mouse, trackpad,gaming console, handheld video game system, gamepad, game controller,desktop computer, e-book reader, camera, video camera, portablemultifunction device such as a cell phone, smartphone, personal digitalassistant (PDA), laptop, tablet computer, digital music player, etc.

In the example shown in FIG. 1B, the haptic output devices 240 and 280are in communication with the processor 220 and are configured toprovide one or more haptic effects. For example, when the processor 220transmits an actuation signal to haptic output device 240, haptic outputdevice 280, or both, the respective haptic output device(s) 240, 280outputs a haptic effect based on the actuation signal. In some examples,the processor 220 is configured to transmit a haptic output signal tohaptic output device 240 comprising an analog drive signal. In otherexamples, the processor 220 is configured to transmit a high-levelcommand to haptic output device 280. The command can include a commandidentifier and zero or more parameters to be used to generate anappropriate drive signal to cause the haptic output device 280 to outputthe haptic effect. Different signals and different signal types may besent to each of one or more haptic output devices. For example, aprocessor may transmit low-level drive signals to drive a haptic outputdevice 240 to output a haptic effect. Such a drive signal may beamplified by an amplifier or may be converted from a digital to ananalog signal, or from an analog to a digital signal, using suitableprocessors or circuitry to accommodate the particular haptic outputdevice 240 being driven.

To generate haptic effects, many devices utilize some type of actuatoror haptic output device 240, 280. The haptic output device 240, 280 mayinclude an electromagnetic actuator such as an Eccentric Rotating Mass(“ERM”) in which an eccentric mass is moved by a motor, a LinearResonant Actuator (“LRA”) in which a mass attached to a spring is drivenback and forth, or a “smart material” such as piezoelectric,electro-active polymers or shape memory alloys. Haptic output devicesalso broadly include other devices (e.g., non-mechanical andnon-vibratory devices) such as those that use electrostatic friction(ESF), ultrasonic surface friction (USF), or those that induce acousticradiation pressure with an ultrasonic haptic transducer, or those thatuse a haptic substrate and a flexible or deformable surface, or thosethat provide projected haptic output such as a puff of air using an airjet, and so on.

In some examples, deformation of one or more components can be used toproduce a haptic effect. For instance, one or more haptic effects may beoutput to change the shape of a surface or a coefficient of friction ofa surface. In an example, one or more haptic effects are produced bycreating electrostatic forces and/or ultrasonic forces that are used tochange friction on a surface. In other examples, an array of transparentdeforming elements may be used to produce a haptic effect, such as oneor more areas comprising a smart gel.

Some examples comprise haptic output devices 240, 280 that are overlaidon the touch-sensitive display 250 or otherwise coupled to thetouch-sensitive display 250. The haptic output devices 240, 280 mayoutput frictional or deformation effects to the touch-sensitive surfaceof the touch-sensitive display 250. In some examples, other portions ofthe system may provide such forces, such as portions of the housing thatmay be contacted by the user or in a separate touch-sensitive inputdevice that is coupled to the system 200. Co-pending U.S. patentapplication Ser. No. 13/092,484, filed Apr. 22, 2011, entitled “Systemsand Methods for Providing Haptic Effects,” the entirety of which ishereby incorporated by reference, describes ways that one or more hapticeffects can be produced and describes various haptic output devices 240,280.

It will be recognized that any type of input synthesis method may beused to generate the interaction parameter from one or more hapticeffect signals including, but not limited to, the method of synthesisexamples listed in TABLE 1 below.

TABLE 1 METHODS OF SYNTHESIS Synthesis Method Description Additivesynthesis combining inputs, typically of varying amplitudes Subtractivesynthesis filtering of complex signals or multiple signal inputsFrequency modulation modulating a carrier wave signal with one orsynthesis more operators Sampling using recorded inputs as input sourcessubject to modification Composite synthesis using artificial and sampledinputs to establish a resultant “new” input Phase distortion alteringthe speed of waveforms stored in wavetables during playback Waveshapingintentional distortion of a signal to produce a modified resultResynthesis modification of digitally sampled inputs before playbackGranular synthesis combining of several small input segments into a newinput Linear predictive similar technique as used for speech synthesiscoding Direct digital computer modification of generated waveformssynthesis Wave sequencing linear combinations of several small segmentsto create a new input Vector synthesis technique for fading between anynumber of different input sources Physical modeling mathematicalequations of the physical characteristics of virtual motion

In FIG. 1B, the communication interface 260 is in communication with theprocessor 220 and provides wired or wireless communications from thesystem 200 to other components or other devices. For instance, thecommunication interface 260 may provide wireless communications betweenthe system 200 and a communications network. In some examples, thecommunication interface 260 may provide communications to one or moreother devices, such as another system 200 and/or one or more otherdevices, such as a television, DVR, or other audio-visual component. Thecommunication interface 260 can be any component or collection ofcomponents that enables the system 200 to communicate with anothercomponent, device, or network. For example, the communication interface260 may comprise a PCI communication adapter, a USB network adapter, oran Ethernet adapter. The communication interface 260 may communicateusing wireless Ethernet, including 802.11 a, g, b, or n standards. Inone example, the communication interface 260 can communicate using RadioFrequency (RF), Bluetooth, CDMA, TDMA, FDMA, GSM, Wi-Fi, satellite, orother cellular or wireless technology. In other examples, thecommunication interface 260 may communicate through a wired connectionand may be in communication with one or more networks, such as Ethernet,token ring, USB, FireWire 1394, fiber optic, etc. In some examples, thesystem 200 comprises a single communication interface 260. In otherexamples, system 200 comprises two, three, four, or more communicationinterfaces.

The example shown in FIG. 1B also comprises a force sensor 290, thoughsome examples may comprise no force sensors or a plurality of forcesensors. Additionally, the force sensor 290 may be housed in the samecomponent as the other components of the system 200 or in a separatecomponent. For example, the processor 220, memory 230, and force sensor290 may all be comprised in the housing 210. The force sensor 290 is incommunication with the processor 220 and is configured to transmit oneor more signals comprising force information to the processor 220. Suchforce information may include a magnitude, a location, an area, a changein force, a rate of change in force, and other types of forceinformation. In some examples, the force sensor 290 may comprise loadcells, force-sensing resistors (FSR), or quantum tunneling composites(QTC), which can, for example, be mounted under a touch-sensitivesurface to measure applied pressure or force. The applied pressure orforce can also be estimated based on changes in the area of contact orcolor of the finger pad. For example, the processor 220 can receive asensor signal from the force sensor 290 indicating an area of contact ofa user's finger pad against the touch-sensitive display 250. Theprocessor 220 can estimate the amount of force being applied to thetouch-sensitive display 250 based on the area of contact (e.g., a largercontact area may be indicative of higher force).

In some examples, the force sensor 290 is configured to sense forceswithin a desirable range. For example, a force sensor 290 is configuredto continuously sense and measure forces in a range of 1 to 200grams-force, with 8-bits or more of resolution. In some examples, theforce sensor 290 may provide more coarse measurements. For example, onesuitable force sensor 290 may be capable of outputting three values: noforce applied, low force, and high force.

Still additional types of force sensors 290 may be suitable for use withexamples according to this disclosure. For example, a force sensor 290can include two switches. The first switch can activate upon a userapplying a force above a first threshold to the force sensor 290. Thesecond switch can activate upon a user applying a force above a secondthreshold to the force sensor 290. Each of the switches can transmitsignals to the processor 220 when activated. In this manner, the forcesensor 290 can detect different amounts of force and transmit associatedsignals to the processor 220. Force sensor 290 may comprise any numberor configuration of switches that actuate in any manner responsive toany number or configuration of force levels.

In some examples, the system 200 may not include a force sensor 290, butmay determine a pseudo force. For instance, a touch-sensitive surfacemay generate a pseudo-force signal based on a contact with thetouch-sensitive surface. The pseudo force in one such example may bebased on the amount of capacitance resulting from a conductor (e.g., theuser's finger) touching the touch-sensitive surface. Accordingly, theamount of capacitance is not a direct measure of force but rather apseudo force. The pseudo force provided by the touch-sensitive surfacemay not be a measure of the actual vertical displacement by a conductorat a single point on the touch-sensitive surface, but rather anestimation of the vertical displacement based on the size of thecapacitance change. The pseudo force may or may not accurately representthe amount of force actually exerted on the touch-sensitive surface. Forexample, the larger the surface of the conductor (e.g., a user's finger)used on the touch-sensitive surface, the larger the change incapacitance per amount of force exerted. If a user presses heavilyagainst the touch-sensitive surface with a fleshy part of the finger,the amount of touch-sensitive surface area covered by the finger isgreater than then when the same part of the finger is touching lightly.In some examples, the area covered, and the corresponding pseudo force,is also greater than when the user presses heavily with a bony part of afinger.

In some examples, additional sensors (not shown) may be incorporatedinto the system 200. For example, sensors may be disposed within thehousing 210 or in a component separate from another component thathouses the memory 230 and/or processor 220. For instance, a wearablesensor may be in communication with the processor 220 and memory 230, auser device, or wearable device via a wired or wireless connection. Insome examples, such sensors may be configured to sense an environmentalfactor, which can represent at least one of an ambient condition or aforce applied to the sensor. The additional sensors can comprise anynumber or type of sensing components. As an example, the sensors maycomprise an accelerometer or gyroscope. A non-limiting list of examplesof sensors and environmental factors is provided below:

TABLE 1 Example Sensors and Factors Sensor Environmental Factor SensedAccelerometer Force in one, two, or three directions Altimeter AltitudeThermometer Ambient temperature; user body temperature Heart ratemonitor Heart rate of device user Skin resistance Skin resistance ofdevice user monitor Oxygen sensor Oxygen use of device user Audiosensor/ Ambient audio and/or audio generated microphone by device userPhotosensor Ambient light IR/Photosensor User eye movement, position,body temperature Hygrometer Relative humidity Speedometer VelocityPedometer/odometer Distance traveled chronometer time of day, date

Environmental factors can include any of the environmental factors notedabove or any other quantities representative of an ambient condition orforce applied to or directed to the device of system 200. Additionally,environmental factors may be evaluated directly from sensor data or maybe processed by the device to derive other environmental factors. Forexample, acceleration data may be used to determine a deviceorientation, velocity and/or a pattern of motion. As a further example,physiological data such as heart rate, skin resistance, and otherfactors can be used to determine a physiological state of a device user(e.g., awake, stressed, asleep, REM sleep, etc.).

In some examples, the system 200 comprises a mouse, trackball, trackpad,joystick, gamepad, or other user interface device. The processor 220,memory 230, force sensor 290, and/or haptic output devices 240, 280 maybe disposed within or coupled to the user interface device. In someexamples, the force sensor 290 can be configured to detect an amount offorce applied to one or more manipulanda (e.g., trackball,touch-sensitive surface, button, or joystick) of the user interfacedevice and transmit an associated sensor signal to the processor 220.

In some examples, the system 200 is configured to be worn by a user. Forexample, the system 200 may be embedded in or coupled to a wearabledevice, such as a wrist watch, other jewelry, gloves, etc. In oneexamples, the system 200 may comprise one or more force sensors 290coupled to the fingertips of a glove. The user may wear the glove andinteract with a surface, such as a table, wall, or desk. The system 200may detect, via the force sensors 290, the user interacting with thesurface and transmit an associated sensor signal to the processor 220.For example, the system 200 may output a (visible or invisible) virtualobject on a display, or project (e.g., via a projector) the virtualobject onto a surface. The system 200 may detect the user contacting thedisplay or surface at a particular location and/or with a particularamount of force to interact with the virtual object. Based on the userinteraction, the system 200 may manipulate a GUI (output on the displayor projected onto the surface), switch between system states or modes,switch between user interface levels, and/or output an associated hapticeffect.

For example, as the user presses a finger against a table with differentamounts of force, the system 200 may switch between user interfacelevels, or between an exploration mode and a manipulation mode. Asanother example, the system 200 may output a haptic effect via hapticoutput devices 240, 280 based on the user interaction. The haptic outputdevices 240, 280 may be coupled to the wearable device, the display, thesurface, or any combination of these. For example, the wearable devicecan include haptic output devices 240, 280 coupled to the fingertips ofthe glove. The wearable device may output haptic effects to the user'sfingertips via the haptic output devices 240, 280. As another example,the system 200 may transmit (e.g., wirelessly) haptic signals to ahaptic output device 280 coupled to the display or surface, causing thedisplay or surface, respectively, to output the haptic effect (e.g., avibration).

Referring now to FIG. 2A, FIG. 2A shows a method 300 for force-basedobject manipulation and haptic sensations according to one example ofthis disclosure. FIG. 2A is described with respect to a softwareapplication executed by the system 200 shown in FIG. 1B; however, themethods disclosed herein are not limited to execution by only the deviceshown in FIG. 1B, but rather may be executed by any suitable systemaccording to this disclosure.

The method 300 begins in block 310 where the system 200 determines afirst threshold value. To determine the first threshold, the system 200may use a default value encoded within the software application, mayread the first threshold from a computer-readable medium, such as from aconfiguration file stored in a hard disk, may receive the firstthreshold from a remote device over a communications network, or mayreceive the first threshold from a user preference setting, such as by auser entering a desired threshold. In some examples, the first thresholdmay be adaptive in that the system 200 uses an initial value for thefirst threshold and, based on usage patterns, adjusts the firstthreshold up or down over time. For instance, the system 200 may adjustthe first threshold up based on how the user uses the device over time.

In some examples, the system 200 may determine a hysteretic firstthreshold. For instance, in the example shown in FIG. 4, the firstthreshold comprises two different force amounts, a low force and a highforce. The different forces are used depending on the mode of the system200. As may be seen in FIG. 4, the system 200 may be in either anexploration mode or a manipulation mode. While the system 200 is in theexploration mode, the first threshold corresponds to the high forcevalue, identified as the “manipulation threshold” in FIG. 4. However,while the system 200 is in the manipulation mode, the first thresholdcorresponds to the low force value, identified as the “explorationthreshold.” Such a hysteretic threshold may be used to require a user toapply significant force to transition from the exploration mode to themanipulation mode, but allow the user to reduce the force used to a morecomfortable level, without reverting to the exploration mode. In someexamples, a hysteretic threshold may comprise a first force value of 50grams-force and a second force value of 100 grams-force.

In some examples, the first threshold may comprise both a force leveland a duration. In such an example, the system 200 will check the user'sforce level against the force threshold, but the system 200 will onlydetermine that the first threshold has been met or exceeded after theuser maintains the force level for the duration required. Similarly,after satisfying the first threshold, the system 200 may also require,before indicating a drop back below the threshold, that the force dropbelow the first threshold force level for the duration. Such conditionsmay provide a friendlier user interface that reduces user frustrationbased on unintentional gestures. Once the first threshold has beendetermined, the method 300 proceeds to block 320.

In block 320, the system 200 receives one or more signals indicating auser interaction (e.g., contact) with a force-sensitive surface (e.g., atouch-sensitive surface or force-sensitive button), the one or moresignals comprising a first force and/or a first location. For example,as a user touches the touch-sensitive display 250, the force-sensitivesurface may provide one or more signals indicating a location of theuser interaction and an amount of force. In some examples, as discussedabove, the system 200 may include a touch-sensitive display 250 thatprovides a signal indicating a location of a user interaction, and aseparate force sensor 290 that provides force information. Thus,multiple signals may be received and used according to some examples ofthe present disclosure. In some examples, a single signal may includeboth location information and force information. After the one or moresignals are received, the method 300 proceeds to block 330.

In block 330, the system 200 determines whether the user interactioncorresponds with an object displayed on a display screen (e.g.,touch-sensitive display 250). For instance, the system 200 may determinewhether the location of a virtual cursor controllable by the user inputdevice corresponds with an object displayed on the display screen. Insome examples, the system 200 may determine whether the user interactioncorresponds with the object based on the first location. For example, ifthe display screen shows a home screen having a plurality of icons andwidgets, the processor 220 determines whether the user interaction is ata location corresponding to an icon or widget. In some examples, otherobjects may be displayed by the system 200. For example, the system 200may display controls for a user interface system for a car on thedisplay screen, such as the climate control system 402 shown in FIG. 3.In such an example, the system 200 may determine whether the userinteraction corresponds to a control displayed on the display screen,such as an “A/C” control or a defroster control. In another examplecorresponding to a touch-based game, such as the game 404 shown in FIG.3, the system 200 may determine whether the user interaction correspondsto one of the controls for maneuvering the dog on the surfboard. Otherobjects may include an image of a lighter 406, an image of a rock 408,or an image of a feather 410, such as those shown in FIG. 3. Such imagesmay also convey a sense of the pressure or force required to interactwith the object. For example, a user may have an expectation that moreforce may be required to interact with a rock than with a feather, orthat different forces may trigger different results, such as with thelighter. If the system 200 determines that the user interactioncorresponds to an object on the display screen, the method 300 proceedsto block 340, otherwise the method 300 returns to block 320.

In block 340, the system 200 compares the first force to the firstthreshold and determines whether the first force meets or exceeds thefirst threshold. If the first force meets or exceeds the firstthreshold, the method 300 proceeds to block 350. As discussed above, insome examples where hysteretic thresholds are employed, the system 200may use a different value for the first threshold depending on the stateof the system 200. Referring to the state diagram 302 shown in FIG. 2C,if the system 200 is in the exploration mode 380, the first thresholdmay correspond to a first value, while if the system 200 is in themanipulation mode 354, the first threshold may correspond to a secondvalue. In addition, in some examples, the threshold determination mayalso include a timing component, in which the force must meet or exceed(or drop below) the first threshold for a minimum duration. Otherwise,the method 300 proceeds to block 342.

In block 342, the system 200 determines whether to output a hapticeffect. In this example, the system 200 outputs a haptic effect if theuser interaction is at a location corresponding to an object, and willthus proceed to block 344. However, if the user is interacting with adisplayed object, in some examples the method returns to block 320 toreceive additional user interaction signals. In some examples, themethod 300 returns to block 310. The method 300 may return to block 310to re-determine the first threshold based on usage of the device, suchas in the case of an adaptive first threshold.

In block 350, the system 200 outputs a first haptic signal to a firsthaptic output device, the first haptic signal configured to cause afirst haptic effect. In the example shown in FIG. 2A, the first hapticeffect is configured to indicate to the user that the first thresholdhas been reached. In some examples, upon reaching or exceeding the firstthreshold, the system 200 transitions from an “exploration” mode to a“manipulation” mode. Thus, the haptic effect is configured to indicateto the user the change in operational modes of the system 200 oractivation of the object contacted, rather than simply that thethreshold has been met. If the system 200 was already in the explorationmode, the system 200 may determine that no haptic effect should beoutput to indicate the transition, though it may still output a hapticeffect if the user interaction (e.g., the user's contact) corresponds toan object displayed on the touch-sensitive display 250. In someexamples, the haptic effect may instead only be configured to indicatethat the user has begun manipulating the object. After outputting thehaptic effect signal, the method 300 proceeds to block 360.

In block 360, the system 200 determines an action to take with respectto the object based on the user interaction. For example, if the objectis an icon corresponding to an application, the system 200 may determinethe action is to launch the application. Alternatively, the system 200may determine that the user is attempting to move the icon and may “pickup” the icon to be moved based on movement of the user interaction(e.g., contact location). At a later time, if the force drops below thefirst threshold, the system 200 may determine the user is dropping theicon, and may leave the icon in the location corresponding to the userinteraction location.

In the climate control system 402 for a car shown in FIG. 3, the system200 may determine the action includes changing the state of a button, orselecting a new option or setting for a control, such as a temperaturesetting or activating/deactivating an air conditioning control. In thegaming application shown in FIG. 3, the system 200 may determine anaction corresponding to the game, such as to change the orientation ofthe dog on the surfboard, or the surfboard itself. After the system 200determines the action to take, the method 300 proceeds to block 370.

In block 370, the system 200 executes the action. As discussed above,actions such as launching applications, changing settings of controls,and/or inputs into gaming applications may be executed by the system 200in response to determining the action to be taken.

FIG. 2C depicts a state diagram 302 corresponding to the method 300shown in FIG. 2A. As may be seen, the state diagram represents twostates of operation for the system 200, an exploration mode 380 and amanipulation mode 354. Initially, the system 200 is in the explorationmode 380 where the user may contact the display screen at variouslocations, but without exceeding the first threshold, the user'scontacts do not trigger any actions to be taken by the system 200.Rather, the system 200 outputs haptic effects when the user's contactcorresponds to an object displayed on the display screen. However, ifthe user's contact meets or exceeds the first threshold, the system 200transitions to the manipulation mode 354, where the user's contacts thatcorrespond to an object may trigger an action with respect to theobject. Further, if in the manipulation mode 354, the system 200 mayreturn to the exploration mode 380 if the contact force no longersatisfies the first threshold. In some examples, the system 200 mayoutput haptic effects when the system 200 transitions between states,either from the exploration mode 380 to the manipulation mode 354, offrom the manipulation mode 354 to the exploration mode 380. Suchtransition haptic effects may be used to inform the user of the changein operational mode and provide a more intuitive force-based interface.In other examples, the system 200 may output haptic effects just priorto the system 200 changing between states—e.g., as the contact forceapproaches the first threshold. This may indicate to the user that thesystem 200 is about to change states, so the user can avoid changingstates inadvertently.

Referring now to FIG. 2B, FIG. 2B is described with respect to asoftware application executed by the system 200 shown in FIG. 1B;however, the methods disclosed herein are not limited to execution byonly the device shown in FIG. 1B, but rather may be executed by anysuitable system according to this disclosure.

The method 304 of FIG. 2B begins in block 309. In block 309, the system200 determines a first threshold. The description of determining a firstthreshold is provided above with respect to block 310 of FIG. 2A andthat description is incorporated herein by reference. In addition, thesystem 200 also determines a second threshold that is greater than thefirst threshold. In this context, the second threshold is greater thanthe first threshold if the absolute value of the magnitude of the secondthreshold is greater than the absolute value of the magnitude of thefirst threshold. In some examples, the second threshold may comprise ahysteretic threshold. The second threshold may also comprise a duration.After determining the first and second thresholds, the method 304proceeds to block 320.

The descriptions of blocks 320-344 are the same as for FIG. 2A above,and that description is incorporated herein by reference. However, aftercomparing the first force to the first threshold in block 340, themethod 304 proceeds to block 351.

In block 351, the system 200 compares the first force to the secondthreshold. If the first force is less than the second threshold, themethod 304 proceeds to block 350. Otherwise, the method 304 proceeds toblock 352.

In blocks 350, 360, and 370, as discussed above with respect to FIG. 2A,incorporated here by reference, the system 200 outputs a haptic effectsignal, determines an action to take with respect to the object based onthe user interaction, and executes the action, respectively. The method304 then proceeds to block 320.

In block 352, the system 200 outputs a haptic effect signal. This stepcan be performed substantially similarly to block 350 discussed abovewith respect to FIG. 2A. After outputting the haptic effect signal, themethod 304 proceeds to block 361.

In block 361, the system 200 determines an action. This step can beperformed substantially similarly to block 350 discussed above withrespect to FIG. 2A. After determining the action, the method 304proceeds to block 362.

In block 362, the system 200 modifies the action to be taken. Forexample, if the determined action is to launch an applicationcorresponding to an icon, the system 200 may instead determine to modifythe action to move the icon away from the user interaction (e.g., acursor location or the user's contact) and provide a vibratory hapticeffect. Referring again to FIG. 3, if the user is interacting with theclimate control system 402 and contacts the A/C button, the system 200may determine the action is to enable the A/C and also change thetemperature setting to a minimum temperature setting. Or if the user isinteracting with the game 404, the system 200 may determine that theaction should be modified to flip the surfboard over causing the dog tofall into the water. In other examples, the determined action may bemodified in different ways. After modifying the action, the method 304proceeds to block 372.

In block 372, the system 200 executes the modified action. This step canbe performed substantially similarly to block 370 discussed above withrespect to FIG. 2A.

FIG. 2C also depicts a state diagram 306 corresponding to the method 304shown in FIG. 2B. The state diagram 306 represents three modes or statesof operation for the system 200, an exploration mode 380, a manipulationmode 354, and a modified manipulation mode 364. Initially, the system200 is in the exploration mode 380 where the user may contact the screenat various locations, but without exceeding the first threshold, theuser's contacts do not trigger any actions to be taken by the system.Rather, the system 200 outputs haptic effects when the user's contactcorresponds to an object displayed on the screen. However, if the user'scontact meets or exceeds the first threshold, the system 200 transitionsto the manipulation mode 354, where the user's contacts that correspondto an object may trigger an action with respect to the object. If theuser's threshold then further meets or exceeds the second threshold, thesystem 200 transitions to the modified manipulation mode 364, asdescribed above with respect to method 304. Further, if in themanipulation mode 354, the system 200 may return to the exploration mode380 if the contact force no longer satisfies the first threshold.Likewise, if in the modified manipulation mode 364, the system 200 mayreturn to the manipulation mode 354 if the contact force no longersatisfies the second threshold.

Referring now to FIG. 4, FIG. 4 shows a graph of a user's force appliedto a force-sensitive surface (e.g., a force-sensitive button, mouse,trackball, joystick, or touch-sensitive surface) over time in an exampleemploying two modes, an exploration mode and a manipulation mode. Inconsidering the graph shown in FIG. 4 with respect to the method 300shown in FIG. 2A, the first threshold of FIG. 2A in this examplecomprises a hysteretic threshold in which the first threshold has twovalues: one value that is used while the system 200 is in theexploration mode, and a second value that is used while the system 200is in the manipulation mode. In such an example, the user may use theexploration mode until the applied force exceeds the higher of the twovalues for the first threshold, at which point the system 200transitions to the manipulation mode. As the user continues to interactwith the system 200, the force of the user's contact may change and dropbelow the higher of the two values of the first threshold, whileremaining above the lower of the two values of the first threshold.Thus, the user will remain in the manipulation mode despite a contactforce dropping below the upper range of the threshold that triggered thetransition from exploration to manipulation.

Referring now to FIG. 5A, FIG. 5A shows a method 500 for force-basedobject manipulation and haptic sensations according to one example ofthis disclosure. FIG. 5B shows a state diagram 502 associated with themethod 500 of FIG. 5A. FIG. 5A is described with respect to a softwareapplication executed by the system 200 shown in FIG. 1B; however, themethods disclosed herein are not limited to execution by only the deviceshown in FIG. 1B, but rather may be executed by any suitable systemaccording to this disclosure.

The method of FIG. 5A may be advantageously employed to provideforce-based navigation in a user interface having multiple layers. Forexample, as discussed above with respect to some illustrative examples,a user interface for a map application may have multiple different viewsof the same portion of the map. In one example, the map may provide acity map layer, a road map layer, and a terrain map layer. As the userincreases the contact force on a force-sensitive surface (e.g.,touch-sensitive display 250), the user interface may transition betweenlayers of the user interface based on the contact force. This may allowthe user to easily and seamlessly transition to different views withoutthe need to locate menu options or controls to perform the function. Insome examples, the user can customize what information is output at eachlayer of the user interface. For example, the user may be able tocustomize whether a city map, road map, or terrain map is output at aparticular layer of the user interface.

In addition, by transitioning to different layers, the system 200 mayprovide different sets of haptic sensations. For example, in a mappingapplication having multiple layers with different types of visualinformation, the system 200 may switch between different sets of hapticinformation, or may select different haptic effects from a largerlibrary of haptic effects. For example, when the mapping application isdisplaying information regarding the first layer, e.g. a city map layer,the system 200 may provide haptic effects when a user interacts with avirtual city shown on the map via the force-sensitive surface (e.g., bycontacting a location on a touch-sensitive display 250 or via a cursor).Such a haptic effect may include increased friction, such as provided byESF, or may include a pulsed vibration output by an actuator. Forexample, the intensity of the vibration, or the size of the area withincreased friction, may vary with the population of the city or thegeographic size of the city.

In some examples, the system 200 may output haptic effects configured toprovide information about the layer to the user. For instance, thesystem 200 may output a number of vibration pulses corresponding to thelayer number. The user may perceive the pulses and be able to determine,e.g., without visually focusing on the input device or a display, whatlayer of the user interface that the user is interaction with or whatstate the system 200 is in. This may be particularly advantageous if thesystem 200 is in a car, so the user can focus on the road. As anotherexample, the system 200 may output haptic effects configured to providethe user with information about the number of available user interfacelevels. For example, upon a user using the input device to interact withan object, the system 200 may output a number of vibration pulsescorresponding to the number of user interface levels with which the usercan interact. This may provide the user with information about the userinterface that, in some examples, the user may not otherwise know.

If the user increases the force applied to the force-sensitive surface,the system 200 transitions to the second layer of the mappingapplication, which may correspond to a road map layer. In such anexample, the system 200 may output ESF haptic effects when the user'scontact corresponds to a road. Further, the system 200 may generatehaptic effects that correspond to the shape of the roads to allow theuser to tactilely trace roads, or the type of road (unpaved, citystreet, state highway, interstate highway) to allow the user toexperience a richer or more intuitive interaction with the mappingapplication. In some examples, the system may change both the visualdisplay and the haptic effects following the transition to the secondlayer, or may leave the visual display unchanged, but change the hapticeffects to provide effects associated with the second layer. Thus, theuser may continue viewing a satellite map of a region, but by pressingharder, may transition from feeling cities to feeling different roads.

If the user again increases force applied to the force-sensitivesurface, the system 200 transitions to the third layer to provide avisual representation of the terrain for the displayed portion of themap. The system 200 then provides haptic effects that correspond to thedifferent types of terrain displayed. For example, the system 200 maydeform the surface of the force-sensitive surface to feel sharp orprickly to indicate trees, while prairies or open spaces may havereduced friction to feel smooth. Other features, such as hills ormountains may be haptically displayed such as by using bumps of varyingsize, or by outputting vibrations of different frequencies or havingdifferently-timed pulses. Further, and as discussed above, in someexamples, the system may leave the visual display unchanged, but changethe haptic effects to provide effects associated with the third layer.Thus, the user may continue viewing a satellite map of a region, but bypressing harder, may transition from feeling different roads to feelingthe various types of terrain displayed on the map.

Still other examples may include a greater or lesser number of layers,and may use different haptic effects, or different haptic output devicesor combinations of haptic output devices, based on the layer selected bythe user.

In another example, such a layered user interface may be employed in adrawing application to expose user interface cues to aid in the draftingprocess. For instance, by increasing an amount of force on aforce-sensitive surface, the user interface may display aids such ashorizon lines, alignment grids, perspective lines, or in a CAD system,may peel away layers of foreground objects to expose other objects thatare otherwise hidden by objects “nearer” to the user within the drawing.In another example, the user may be able to navigate through differentlayers of three-dimensional images, such as three-dimensional scansresulting from medical procedures such as CT scans or MRIs.

Another example may comprise a multi-layered or three-dimensional userinterface. For example, a user interface may be displayed as a tiled setof icons corresponding to applications, but display additional icons indeeper “planes” or layers of the user interface. To access theseadditional layers, the user may increase force on the force-sensitivesurface. Once the user has accessed the deeper layer, the user mayreduce the force on the force-sensitive surface and remain at the newlayer, or may reduce the force on the force-sensitive surface further toreturn to a prior layer.

Still another example may comprise a multi-layered “circular” userinterface. For example, a user interface for a mapping application mayhave three layers: a city layer, a road layer, and a terrain layer. Eachtime the user applies an amount of force exceeding the threshold to theforce-sensitive surface, the system can transition to the next layer.Upon reaching the last layer, if the user applies an amount of forceexceeding the threshold, the system can transition back to the firstlayer. For example, the user interface for the mapping application mayoutput (e.g., by default) the city layer. The user may apply an amountof force exceeding the threshold to cycle to the road layer. The usermay again apply an amount of force exceeding the threshold to cycle tothe terrain layer. If the user again applies an amount of forceexceeding the threshold, the system can cycle back to the city layer. Inthis manner, the user can cycle or transition through the user interfacelayers by applying an amount of force exceeding the threshold.

The method 500 of FIG. 5 begins at block 510 where the system 200determines first and second thresholds. The description of determiningthe first and second thresholds is provided above with respect to block309 of FIG. 2B, which is incorporated here by reference. After thesystem 200 determines the first and second thresholds, the method 500proceeds to block 520.

In block 520, the system 200 receives one or more signals indicating auser interaction with a force-sensitive surface, the one or more signalscomprising a first force and/or a first location. The force-sensitivesurface can be in communication with a processor-based device, and theprocessor-based device can be in communication with a display screen andproviding a graphical user interface (GUI) having a plurality of layers.The display screen can display a first layer of the GUI.

In block 530, the system 200 compares the first force to the firstthreshold. If the first force meets or exceeds the first threshold, themethod proceeds to block 540. Otherwise, the method returns to block520.

In block 540, the system 200 generates a signal configured to cause asecond layer of the GUI to be displayed on the display screen, andtransmits the signal to cause the second layer to be displayed. Afterchanging to the second layer, if there are additional layers, the methodproceeds to block 550.

In block 550, the system 200 compares the first force to the secondthreshold. If the first force meets or exceeds the second threshold, themethod proceeds to block 560. Otherwise, the system remains at thesecond layer, as depicted in block 555.

In block 550, the system 200 generates a signal configured to cause athird layer of the GUI to be displayed on the display screen, andtransmits the signal to cause the third layer to be displayed.

In some examples, more than three layers may be employed. In some suchexamples, additional thresholds may be used to trigger transitions toadditional layers.

FIG. 5B depicts a state diagram 502 depicting the states and transitionsbetween the states corresponding to the method 500 shown in FIG. 5A. Thestate diagram 502 represents three different layers for the GUI ofsystem 200: Layer 1 state 532, Layer 2 state 534, and Layer 3 state 552.Initially, the system 200 is in the Layer 1 state 532 where the user maycontact the display screen at various locations, but without exceedingthe first threshold, the user's actions do not trigger a change toanother layer. However, if the user's contact meets or exceeds the firstthreshold, the system 200 transitions to the Layer 2 state 534. If theuser's threshold then further meets or exceeds the second threshold, thesystem transitions to the Layer 3 state 552, as described above withrespect to method 304.

Referring now to FIG. 6, FIG. 6 shows a graph of a user's force appliedto a force-sensitive surface over time in an example employing a userinterface for a mapping application that has three layers: a city layer,a road layer, and a terrain layer. The graph shown in FIG. 6 correspondsto one example of the method 500 shown in FIG. 5A. In this example, thefirst and second thresholds of FIG. 5A each comprise a hystereticthreshold wherein the thresholds each have two values: one value that isused while the system 200 is in a higher layer, and a second value thatis used while the system 200 is in a lower layer. In such an example,the user may interact with the city layer of the user interface untilthe applied force exceeds the higher of the two values for the firstthreshold, at which point the system 200 transitions to the road layerof the mapping application. As the user continues to interact with thesystem 200, the force of the user's contact may increase above thesecond threshold, causing the system 200 to transition to the terrainlayer of the mapping application.

As the user continues to navigate within the mapping application, theforce applied by the user to the force-sensitive surface may drop belowthe second threshold, returning the system to the road layer of themapping application. And ultimately, the user reduces the force belowthe first threshold, and the system 200 transitions the mappingapplication to the city layer.

In some examples, the first threshold and second threshold can beconfigured such that a force range (a force gap) is between the firstthreshold and the second threshold. The system 200 can output hapticeffects if the user applies an amount of force that falls within thisforce range. For example, if the force applied by the user falls betweenthe force threshold for the city layer and the force threshold for theroad layer, the system 200 can output a haptic effect comprising apulse. This may increase the stability of the system 200, as it mayalert that user that the she is about to change layers and/or prevent auser from inadvertently switching between user interface layers.

Referring now to FIG. 7, FIG. 7 shows a method 700 for force-basedobject manipulation and haptic sensations according to one example ofthis disclosure. FIG. 7 is described with respect to a softwareapplication executed by the system 200 shown in FIG. 1B; however, themethods disclosed herein are not limited to execution by only the deviceshown in FIG. 1B, but rather may be executed by any suitable systemaccording to this disclosure.

The method 700 of FIG. 7 begins in block 710 when the system 200receives one or more signals indicating a user interaction with aforce-sensitive surface, the one or more signals comprising a firstforce and/or a first location. For example, as a user touches thetouch-sensitive display 250, the touch-sensitive display 250 may provideone or more signals indicating a location of the contact and an amountof force. In some examples, as discussed above, the system 200 mayinclude a touch-sensitive display 250 that provides a signal indicatinga location of a contact, and a separate force sensor 290 that providesforce information. Thus, multiple signals may be received and usedaccording to some examples. In some examples, a single signal mayinclude both location information and force information. After the oneor more signals are received, the method 700 proceeds to block 720.

In block 720, the system 200 determines whether the user interactioncorresponds with an object displayed on a display screen (e.g., based onthe first location or the location of an associated cursor). Forexample, if the display screen shows a home screen having a plurality oficons and widgets, the system 200 determines whether an interaction witha touch-sensitive display 250 is at a location corresponding to an iconor widget. In some examples, other objects may be displayed by thesystem 200. For instance, the system 200 may display controls for a userinterface system for a car on the display screen, such as the climatecontrol system 402 shown in FIG. 3. In such an example, the system 200may determine whether the user interaction corresponds to a controldisplayed on the screen, such as an “A/C” control or a defrostercontrol. In another example corresponding to a touch-based game, such asthe game 404 shown in FIG. 3, the system 200 may determine whether theuser interaction corresponds to one of the controls for maneuvering thedog on the surfboard. Other objects may include an image of a lighter406, an image of a rock 408, or an image of a feather 410, such as thoseshown in FIG. 3. After determining that the user interaction correspondsto an object, the method 700 proceeds to block 730.

At block 730, the system 200 generates a first haptic signal (e.g.,based on the first force and a texture associated with the object)configured to cause a haptic output device 240, 280 to output a firsthaptic effect having a first intensity, and outputting the first hapticsignal to the haptic output device 240, 280. For example, icons,widgets, or other graphical objects may be associated with a texture. Asa user draws a finger across a graphical object, she may feel atexture-based haptic effect. In this example, the texture sensationvaries based on the force exerted on the force-sensitive surface by theuser. Thus, if the user only lightly contacts the touch-sensitivedisplay 250 at a location corresponding to a graphical object, she mayfeel only the edges of the object (e.g., edge haptic effects) but littleor no tactile information relating to the object's texture. However, asthe user increases the contact force, the system 200 generates andoutputs haptic effects that are increasingly more representative of thetexture associated with the object.

For example, referring to the images shown in FIG. 3, a user touchingthe rock may feel only the general shape of the rock when pressinglightly against the touch-sensitive display 250. However, if the userincreases the contact force, the user may feel the roughness of portionsof the rock and the smoothness of other portions. And if the usercontinues to increase the contact force, she may feel fine details aboutthe texture of the rock, such as small cracks or bumps on the rock, orthe graininess of rock. Further, by increasing the force on the rock,the apparent size of the rock (visually, tactilely, or both) mayincrease to allow the user to tactilely explore the surface features ofthe rock in more detail by further increasing the force on the rock. Inthis example, the level of detail of the haptic sensations increasesapproximately linearly with the increase in contact force. In otherexamples, the level of detail of the haptic sensations may not increaselinearly. For instance, the level of detail of the haptic sensations mayincrease logarithmically. In some examples, a texture may have differenttexture “layers,” similar to the user interface layers discussed abovewith respect to FIGS. 5-6, such that as the user's force increases, itreaches thresholds at which a different texture layer is presentedthrough haptic effects.

Some examples may provide additional haptic texture effects based onuser interaction. For example, referring to FIG. 8A, two differenthaptically-enabled software applications 810, 820 are shown. The firstsoftware application 810 presents the user with multiple differentanimals with different types of skin or fur that the user can interactwith tactilely to feel the different textures. For example, if the userinteracts with the sheep via the force-sensitive surface, the system 200may output a deformation effect that feels soft or malleable to the userto emulate the sheep's wool, or a rough texture for the wolf's wirycoat, or a smooth or slippery (e.g., low friction) texture for thegoldfish, or a ridged texture for the armadillo's shell.

In addition, the system 200 is configured to increase the intensity ofthe effect as the user presses harder against the force-sensitivesurface (e.g., touch-sensitive display 250). For example, the system 200may employ the following mapping function to adjust the intensity of ahaptic effect:

V( . . . )=k(P)·F( . . . )   Equation 1:

In Equation 1, V corresponds to the output voltage for a haptic outputdevice 240, 280, F( . . . ) identifies a texture or other renderingeffect, and k(P) provides a mapping function that maps the force P to avalue between 0.0 and 1.0. In some examples, a different function, suchas the following function, may be used:

V( . . . )=k(P)·F( . . . )+(1−k(P))·G( . . . )   Equation 2:

In Equation 2, V corresponds to the output voltage for a haptic outputdevice 240, 280, F( . . . ) and G( . . . ) provide textures or otherrendering effects, and k(P) provides a mapping function that maps theforce P to a value between 0.0 and 1.0. Other examples may provideadditional texture or rendering effect functions, or different weightedrelationships between those effect functions.

In the second software application 820 shown in FIG. 8A, the user may beable to manipulate the displayed saw to cut through either or bothropes. In one example, to manipulate the saw, the user may press on thesaw and move her finger back and forth in a sawing motion. In anotherexample, to manipulate the saw, the user may move the force-sensitivesurface (e.g., a mouse or joystick) back and forth in a sawing motionwhile interacting with a manipulandum (e.g., a mouse or joystick button)with a particular amount of force. This may move the virtual saw backand forth. The system 200 provides vibrational or frictional effectsbased on the speed at which the user saws, but also based on the forceexerted by the user on the force-sensitive surface. As the userincreases the force applied to the force-sensitive surface, the saw maycut more deeply into the rope, and thus the user may feel increasedfriction, or increased vibration to indicate that the saw is exertingmore force or that more fibers of the rope are being cut. Examplesaccording to the second software application 820 may also employ mappingequations like those used with respect to the first software application810 discussed above. And while this software application incorporatesthe use of saws, other applications, such as drawing or paintingapplications, or other applications employing tools, may provide similareffects and responses based on the user's manipulation of the respectivetool (e.g., a pen or paintbrush) as well as the force applied to thetool.

Referring now to FIG. 8B, three additional software applications 850,860, 870 for providing force-based object manipulation and hapticsensations are shown. Software application 850 initially displays animage of Bigfoot encased in ice or frost, and the user is able to freehim by scraping away the frost. In some examples, as the user lightlydrags her finger back and forth across the screen, no frost is scrapedaway, but the user may feel a smooth uniform texture or reduced frictionto emulate sliding her finger over ice. If she slightly increases theforce to scrape away the frost, she feels a mild frictional forceprovided by an ESF output device as well as a low-magnitude vibrationprovided by an actuator to emulate the feel of her finger scraping awaythe frost. As the user increases the force of the contact, the system200 employs a mapping function, such as shown in Equation 2 above, toincrease the intensity of the haptic effects, which may also triggeraspects of the haptic effects that are only presented when sufficientforce is applied, such as irregularities in the frost. For example, F( .. . ) may correspond to the frictional effect while G( . . . ) maycorrespond to the vibrational effect. In addition, a greater quantity offrost may be removed.

In software application 860, the system 200 displays to the user anapparently uniform field of sand, which is overlaid on a hard, ridgedsurface. That is, a lower virtual object (e.g., the hard, rigid surface)can be layered beneath an upper virtual object (e.g., the field ofsand). In some examples, as the user interacts with (e.g., lightly dragsher finger over) the surface, the system 200 generates and outputs ahaptic effect associated with a characteristic (e.g., a texture) of theupper virtual object. For example, the system 200 may generate andoutput a texture effect associated with the field of sand, such as ahaptic effect comprising a grainy or gritty texture. Such effects may beprovided by an ESF or deformable haptic output device. If the userapplies force with a greater intensity to the surface, the system 200may generate and output one or more haptic effects associated with thelower virtual object. For example, as the user increases the contactforce, the underlying hard surface becomes apparent from a change in thetexture effect. In this example, rather than simply changing the scalingfactor for, e.g., Equation 1, instead the system 200 adds a secondhaptic texture, such as by transitioning to Equation 2. In someexamples, though, the system 200 may always use Equation 2, but insteadmay provide zero weight to the texture effect corresponding to the hardsurface below a minimum force. Thus, the user is able to, tactilely,determine the existence of information that is not presented visually,i.e., the existence of the hard surface beneath the sand.

The software application 870 provides functionality similar to that ofthe software application 860 in that the user is initially presentedwith a first texture. In this example, the first texture may be adeformation effect that provides a soft or squishy feeling to emulatethe feel of gelatin. However, in certain areas within the gelatin, ahard object may be present, such as the bean shown in FIG. 8B. In somecases, the object may be visible, while in others it may not. In someexamples, as the user drags her finger over the display screen andencounters the bean, the nature of the haptic effect may change toindicate the presence of the bean. For example, as the user presses downwith increasing force, the system 200 outputs a firmer deformation toindicate the presence of the bean surrounded by the softer deformationcorresponding to the gelatin. In some examples, the size of the firmerdeformation may correspond to the size of the bean, while in otherexamples, so long as the user's contact corresponds to the location ofthe bean, the firmer deformation effect is provided.

Referring now to FIG. 9, FIG. 9 shows a method 900 for force-basedobject manipulation and haptic sensations according to one example ofthe present disclosure. FIG. 9 is described with respect to a softwareapplication executed by the system 200 shown in FIG. 1B; however, themethods disclosed herein are not limited to execution by only the deviceshown in FIG. 1B, but rather may be executed by any suitable systemaccording to this disclosure.

The method 900 shown in FIG. 9 begins in block 910 when the system 200receives one or more signals indicating a user interaction with aforce-sensitive surface, the one or more signals comprising a firstforce and/or a first location. The signals may be from thetouch-sensitive display 250 or the force sensor 290. For example, as auser touches the touch-sensitive display 250, the force-sensitivesurface may provide one or more signals indicating a location of thecontact and an amount of force of the contact. In some examples, asdiscussed above, the system 200 may include a touch-sensitive display250 that provides a signal indicating a location of a contact, and aseparate force sensor 290 that provides force information. Thus,multiple signals may be received and used by the system 200. In someexamples, a single signal may include both location information andforce information. After the one or more signals are received, themethod 900 proceeds to block 920.

In block 920, the system 200 determines an interaction between the userand the system 200. Such interactions may include virtually anyinteraction of the user with the system 200. For example, a user maydesire to move an icon from one location to another on thetouch-sensitive display 250. In such an example, the user may press onan icon located on the touch-sensitive display 250. As the userincreases the force on the touch-sensitive display 250, the user'scontact may pass a threshold at which the system 200 determines the useris attempting to move the icon. The method 900 then proceeds to block930.

At block 930, after determining that the user is attempting to move theicon, the system 200 outputs a haptic effect, such as a brief detenteffect. The method 900 then returns to block 910, and proceeds again toblock 920. In the second iteration of block 920, the user moves herfinger to a new location on the screen. In this case, because the userhas “picked up” the icon, the system 200 moves the icon to the newlocation by following the user's finger motion. The method 900 againproceeds to block 930. At block 930, the system 200 provides hapticeffects to facilitate movement of the icon, such as reducing africtional force on the touch-sensitive display 250 (e.g., via an ESF orUSF haptic output device). Again, the method returns to block 910.

At block 910, the system 200 continues to receive user interactionsignals. This may result from the user's continued contact with theforce-sensitive surface (e.g., the touch-sensitive display 250), andproceeds to block 920. However, at this time, the user reduces the forceon the force-sensitive surface, but does not lift her finger entirelyoff of the force-sensitive surface, to “drop” the icon at the newlocation. The system 200 determines that the user's contact force hasdropped below the threshold to permit moving the icon, and the system200 determines that the icon is being dropped. The method 900 proceedsto block 930, where the system 200 outputs another haptic effect, suchas a brief detent. This may indicate the completion of the moveoperation. Thereafter, the method 900 again returns to block 910 toawait continued interaction with the system 200.

The example above is exemplary of use of force-based interactions withsystems according to this disclosure. However, other force-basedinteractions are contemplated. For example, a user may use force tonavigate a cursor within a three-dimensional environment. In one suchexample, lateral movement on the touch-sensitive display 250 maycorrespond to X- and Y-axis displacements within the environment, whileincreased or decreased force may result in a rate-based displacement ina Z-axis. To differentiate between positive and negative movements alongthe Z-axis, the system 200 may output different vibrational or otherhaptic effects. Or the system 200 may output a haptic effect at amidpoint force at which no Z-axis movement occurs to aid the user inmaintaining the Z-axis position, while moving laterally in the X- orY-axes.

In a further example, the system 200 may provide an enhanced capabilityfor unlocking the device after it has sat idle for an extended period oftime. For instance, after a period of inactivity, the system 200 maylock itself to prevent unauthorized use. To unlock the device, the usermay enter a passcode. Alternative, the user may perform a force-basedgesture. For example, the user may swipe her finger across the screenfrom left to right and press heavily three times in rapid succession tounlock the device. Or such a force-based gesture may be combined with atraditional motion-based gesture. For example, the user may swipe azigzag pattern across the touch-sensitive display 250, and press heavilyduring each change in direction, to unlock the device.

In addition, such an unlocking mechanism may incorporate haptics toassist the user's use of force. For example, when performing theforce-enhanced swipe gesture, one example may provide one of threedifferent haptic textures corresponding to three different force level“bands.” In this example, the textures increase in “grittiness” as theuser transitions from a lower force band to a higher force band. In someexamples, the system may output a brief detent to indicate the change inforce bands. Thus, the user is aided by the device as she attempts toapply the appropriate amounts of force. Such an example may also aid theuser in remembering the force sequence as a sequence of tactileresponses, or even as a sequence of numbers corresponding to thedifferent force bands. For example, the user's force code may bemnemonically recalled as 1,2,3,2,3,1, to correspond to each of the 3force bands. Although three force bands are used in this example, someexamples may employ a greater number or lesser number of force bands.

In some examples, contact force may be an integral part of playing videogames. For instance, a user playing a golf game may swing a club byfirst swiping in one direction to create a backswing, and then swing inthe opposite direction to swing the club. To enhance the game, the usermay also be required during the swing to press hard at the moment theuser's finger passes over the ball to strike the ball. The moreaccurately the user's high-force contact corresponds to the location ofthe ball, the more accurate the ball strike may be, or the longerdistance the drive may be. To assist the user in performing the swing,examples may provide haptic effects corresponding to aspects of the golfswing. For instance, the haptic effect may provide tactile feedback tothe user based on the “quality” of the swing. In one such example, asthe user draws the club head back in the backswing, the user may berequired to follow a particular path, or apply appropriate force atparticular times. The system 200 may output haptic effects, such asreducing friction along the correct backswing path, and providingtextures at force points. Further, at the force points, the system 200may output haptic effects configured to assist the user in applying thecorrect amount of force. For example, upon reaching a force point, theuser may increase force until the system 200 outputs a detent, at whichpoint the user may continue the backswing motion. Further, during theswing and contact with the ball, the user may increase force to strikethe ball, and may receive immediate tactile feedback, such as avibrational effect that varies in magnitude based on the accuracy of theforce applied to the ball and the location of the force.

In addition, as with the backswing, the system 200 may provide a textureeffect at the location of the ball to aid the user in applying increasedforce at the proper location. Further, the system 200 may provide one ormore force-activated vibration detents at the ball corresponding to theamount of power to strike the ball with. Thus, the user may bothinteract with the system 200 by applying varying levels of force, butmay also receive haptic feedback both to assist the user in applying theappropriate amount of force, as well as to guide the user whileperforming certain tasks.

In addition to allowing the user to incorporate force changes intointeractions with systems 200 according to the present disclosure, someexamples may also assist the user during such force-based manipulations.For example, as the user increases the contact force with thetouch-sensitive display 250, the system 200 may output a haptic effectconfigured to reduce the coefficient of sliding friction. This may allowthe user to more easily drag her finger across the device. In oneexample, the system 200 increases the haptic effect in direct proportionto the force applied to the device by the user. Such functionality mayallow the user to more easily apply higher force to the device withoutalso applying increased lateral force to allow her finger to move acrossthe surface of the device. In other examples, the system 200 may changethe magnitude of the haptic effect by applying the contact force toanother algorithm, such as a logarithmic scale. In some examples, thesystem 200 may apply a changing texture to the touch-sensitive display250 to indicate the increasing forces sensed by the device. Suchchanging texture may indicate to the user that she may potentially entera different interaction mode, such as a manipulation mode describedabove, if she continues to increase the contact force.

While the methods and systems herein are described in terms of softwareexecuting on various machines, the methods and systems may also beimplemented as specifically-configured hardware, such asfield-programmable gate array (FPGA) specifically to execute the variousmethods. For instance, examples can be implemented in digital electroniccircuitry, or in computer hardware, firmware, software, or in acombination thereof. In one example a device may comprise a processor orprocessors. The processor comprises a computer-readable medium, such asa random access memory (RAM) coupled to the processor. The processorexecutes computer-executable program instructions stored in memory, suchas executing one or more computer programs for editing an image. Suchprocessors may comprise a microprocessor, a digital signal processor(DSP), an application-specific integrated circuit (ASIC), fieldprogrammable gate arrays (FPGAs), and state machines. Such processorsmay further comprise programmable electronic devices such as PLCs,programmable interrupt controllers (PICs), programmable logic devices(PLDs), programmable read-only memories (PROMs), electronicallyprogrammable read-only memories (EPROMs or EEPROMs), or other similardevices.

Such processors may comprise, or may be in communication with, media,for example computer-readable media, that may store instructions that,when executed by the processor, can cause the processor to perform thesteps described herein as carried out, or assisted, by a processor.Examples of computer-readable media may comprise, but are not limitedto, an electronic, optical, magnetic, or other storage device capable ofproviding a processor, such as the processor in a web server, withcomputer-readable instructions. Other examples of media comprise, butare not limited to, a floppy disk, CD-ROM, magnetic disk, memory chip,ROM, RAM, ASIC, configured processor, all optical media, all magnetictape or other magnetic media, or any other medium from which a computerprocessor can read. The processor, and the processing, described may bein one or more structures, and may be dispersed through one or morestructures. The processor may comprise code for carrying out one or moreof the methods (or parts of methods) described herein.

The foregoing description of some examples of the invention has beenpresented only for the purpose of illustration and description and isnot intended to be exhaustive or to limit the invention to the preciseforms disclosed. Numerous modifications and adaptations thereof will beapparent to those skilled in the art without departing from the spiritand scope of the invention.

Reference herein to “one example” or “an example” means that aparticular feature, structure, operation, or other characteristicdescribed in connection with the example may be included in at least oneimplementation of the invention. The invention is not restricted to theparticular examples described as such. The appearance of the phrase “inone example” or “in an example” in various places in the specificationdoes not necessarily refer to the same example. Any particular feature,structure, operation, or other characteristic described in thisspecification in relation to “one example” may be combined with otherfeatures, structures, operations, or other characteristics described inrespect of any other example.

That which is claimed is:
 1. A system comprising: a processor; a hapticoutput device communicatively coupled to the processor; and a memorycommunicatively coupled to the processor, the memory comprising programcode that is executable by the processor to cause the processor to:display a virtual object on a touch-screen display; detect a contact ata location corresponding to the virtual object via the touch-screendisplay; determine an action to take with respect to the virtual objectin response to determining that a force of the contact exceeds a firstforce threshold; and in response to determining that the force of thecontact exceeds a second force threshold that is greater than the firstforce threshold: modify the action to create a modified action, themodified action comprising moving the virtual object in a virtualenvironment; execute the modified action; and use the haptic outputdevice to output a haptic effect associated with the modified action. 2.The system of claim 1, wherein the memory further comprises program codethat is executable by the processor to cause the processor to: execute afirst action in response to determining that the force of the contactexceeds a third force threshold; use the haptic output device to outputa first haptic effect associated with the first action; subsequent toexecuting the first action, execute a second action in response todetermining that the force of the contact has fallen below the thirdforce threshold; and use the haptic output device to output a secondhaptic effect associated with the second action.
 3. The system of claim2, wherein the first action comprises virtually grabbing the virtualobject in a drag-and-drop operation, and the second action comprisesvirtually releasing the virtual object in the drag-and-drop operation.4. The system of claim 1, wherein the action is a first action, andwherein the memory further comprises program code that is executable bythe processor to cause the processor to: detect a pattern of varyingforces applied to the touch-screen display, the pattern of varyingforces comprising a plurality of forces with different magnitudes; inresponse to detecting each respective force of the plurality of forces,use the haptic output device to output one or more haptic effectsconfigured to convey information about the respective force; compare thepattern of varying forces to a predefined pattern of varying forces; andexecute a second action in response to determining that the pattern ofvarying forces matches the predefined pattern of varying forces.
 5. Thesystem of claim 4, wherein the second action comprises unlocking adevice.
 6. The system of claim 1, wherein the memory further comprisesprogram code that is executable by the processor to cause the processorto: generate a three-dimensional virtual environment in which a cursorcan move along an X axis, a Y axis, and a Z axis; navigate the cursoralong the X axis or the Y axis in response to detecting a lateralmovement of an object along a touch-screen display; navigate the cursoralong the Z axis in response to detecting a change in force on thetouch-screen display; and use the haptic output device to output one ormore haptic effects in response to the cursor being navigated along theZ axis, the one or more haptic effects being configured to indicatewhether the navigation is in a positive direction or a negativedirection along the Z axis.
 7. The system of claim 1, wherein thevirtual object is part of a video game, and wherein the memory furthercomprises program code that is executable by the processor to cause theprocessor to: manipulate the virtual object and output a first hapticeffect in response to determining that the force of the contact exceedsa third force threshold; and further manipulate the virtual object andoutput a second haptic effect in response to determining that the forceof the contact exceeds a fourth force threshold that is higher than thethird force threshold.
 8. The system of claim 1, wherein thetouch-screen display is part of a vehicle, and the virtual object is acontrol object for controlling an electronic device in the vehicle. 9.The system of claim 1, wherein the memory further comprises program codethat is executable by the processor to cause the processor to: use thehaptic output device to output haptic effects that are configured tosimulate a texture of the virtual object with (i) increasing intensityin response to the force increasing or (ii) decreasing intensity inresponse to the force decreasing, the haptic effects being generatedaccording to a relationship between the force and the intensity.
 10. Amethod comprising: displaying, by a processor, a virtual object on atouch-screen display; detecting, by the processor, a contact at alocation corresponding to the virtual object via the touch-screendisplay; determining, by the processor, an action to take with respectto the virtual object in response to determining that a force of thecontact exceeds a first force threshold; and in response to determiningthat the force of the contact exceeds a second force threshold that isgreater than the first force threshold: modifying, by the processor, theaction to create a modified action, the modified action comprisingmoving the virtual object in a virtual environment; executing, by theprocessor, the modified action; and using, by the processor, a hapticoutput device to output a haptic effect associated with the modifiedaction.
 11. The method of claim 10, further comprising: executing afirst action in response to determining that the force of the contactexceeds a third force threshold; using the haptic output device tooutput a first haptic effect associated with the first action;subsequent to executing the first action, executing a second action inresponse to determining that the force of the contact has fallen belowthe third force threshold; and using the haptic output device to outputa second haptic effect associated with the second action.
 12. The methodof claim 10, wherein the action is a first action, and furthercomprising: detecting a pattern of varying forces applied to thetouch-screen display, the pattern of varying forces comprising aplurality of forces with different magnitudes; in response to detectingeach respective force of the plurality of forces, using the hapticoutput device to output one or more haptic effects configured to conveyinformation about the respective force; comparing the pattern of varyingforces to a predefined pattern of varying forces; and executing a secondaction in response to determining that the pattern of varying forcesmatches the predefined pattern of varying forces.
 13. The method ofclaim 10, further comprising: generating a three-dimensional virtualenvironment in which a cursor can move along an X axis, a Y axis, and aZ axis; navigating the cursor along the X axis or the Y axis in responseto detecting a lateral movement of an object along a touch-screendisplay; navigating the cursor along the Z axis in response to detectinga change in force on the touch-screen display; and using the hapticoutput device to output one or more haptic effects in response to thecursor being navigated along the Z axis, the one or more haptic effectsbeing configured to indicate whether the navigation is in a positivedirection or a negative direction along the Z axis.
 14. The method ofclaim 10, wherein the virtual object is part of a video game, andfurther comprising: manipulating the virtual object and output a firsthaptic effect in response to determining that the force of the contactexceeds a third force threshold; and further manipulating the virtualobject and output a second haptic effect in response to determining thatthe force of the contact exceeds a fourth force threshold that is higherthan the third force threshold.
 15. The method of claim 10, furthercomprising: using the haptic output device to output haptic effects thatare configured to simulate a texture of the virtual object with (i)increasing intensity in response to the force increasing or (ii)decreasing intensity in response to the force decreasing, the hapticeffects being generated according to a relationship between the forceand the intensity.
 16. A non-transitory computer-readable mediumcomprising program code that is executable by a processor to cause theprocessor to: display a virtual object on a touch-screen display; detecta contact at a location corresponding to the virtual object via thetouch-screen display; determine an action to take with respect to thevirtual object in response to determining that a force of the contactexceeds a first force threshold; and in response to determining that theforce of the contact exceeds a second force threshold that is greaterthan the first force threshold: modify the action to create a modifiedaction, the modified action comprising moving the virtual object in avirtual environment; execute the modified action; and use a hapticoutput device to output a haptic effect associated with the modifiedaction.
 17. The non-transitory computer-readable medium of claim 16,further comprising program code that is executable by the processor tocause the processor to: execute a first action in response todetermining that the force of the contact exceeds a third forcethreshold; use the haptic output device to output a first haptic effectassociated with the first action; subsequent to executing the firstaction, execute a second action in response to determining that theforce of the contact has fallen below the third force threshold; and usethe haptic output device to output a second haptic effect associatedwith the second action.
 18. The non-transitory computer-readable mediumof claim 16, wherein the action is a first action, and furthercomprising program code that is executable by the processor to cause theprocessor to: detect a pattern of varying forces applied to thetouch-screen display, the pattern of varying forces comprising aplurality of forces with different magnitudes; in response to detectingeach respective force of the plurality of forces, use the haptic outputdevice to output one or more haptic effects configured to conveyinformation about the respective force; compare the pattern of varyingforces to a predefined pattern of varying forces; and execute a secondaction in response to determining that the pattern of varying forcesmatches the predefined pattern of varying forces.
 19. The non-transitorycomputer-readable medium of claim 16, further comprising program codethat is executable by the processor to cause the processor to: generatea three-dimensional virtual environment in which a cursor can move alongan X axis, a Y axis, and a Z axis; navigate the cursor along the X axisor the Y axis in response to detecting a lateral movement of an objectalong a touch-screen display; navigate the cursor along the Z axis inresponse to detecting a change in force on the touch-screen display; anduse the haptic output device to output one or more haptic effects inresponse to the cursor being navigated along the Z axis, the one or morehaptic effects being configured to indicate whether the navigation is ina positive direction or a negative direction along the Z axis.
 20. Thenon-transitory computer-readable medium of claim 16, wherein the virtualobject is part of a video game, and further comprising program code thatis executable by the processor to cause the processor to: manipulate thevirtual object and output a first haptic effect in response todetermining that the force of the contact exceeds a third forcethreshold; and further manipulate the virtual object and output a secondhaptic effect in response to determining that the force of the contactexceeds a fourth force threshold that is higher than the third forcethreshold.
 21. The non-transitory computer-readable medium of claim 16,further comprising program code that is executable by the processor tocause the processor to: use the haptic output device to output hapticeffects that are configured to simulate a texture of the virtual objectwith (i) increasing intensity in response to the force increasing or(ii) decreasing intensity in response to the force decreasing, thehaptic effects being generated according to a relationship between theforce and the intensity.